Analytic performance models for single class and multiple class multithreaded software servers

Modern computer systems are based on a wide variety of software servers, such as web servers, application servers, database servers, and mail servers. The typical software architecture of such servers includes a set of processes or threads that serve requests submitted to the server. Requests that arrive at the server and find all threads busy, are placed in a queue. Threads that are busy executing requests compete for hardware resources (e.g., processors and I/O devices) at the machine where the software server runs. It is important to be able to model software servers in a way that takes into account both software contention—waiting for threads—and hardware contention—waiting for processors and I/O devices. This paper presents analytical models for a wide range of multithreaded software server architectures: a) single class (i.e., all requests have similar demands) with unlimited thread queue size, b) single class with limited thread queue size, c) multiclass with unlimited thread queue size, and d) multiclass with limited queue size. Numerical results are presented to illustrate the use of the models.